Control device, control method, and microscope device for operation

ABSTRACT

To make it possible to improve user convenience, provided is a control device including: a control unit configured to control a position and an attitude of a microscope unit by driving an arm unit that supports the microscope unit on the basis of a captured image of an operating site photographed by the microscope unit during an operation so that a position and attitude condition set before the operation is satisfied. The position and attitude condition is a condition that prescribes a position and an attitude of the microscope unit with respect to the operating site to obtain a desired captured image corresponding to the position and attitude condition.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/556,367, filed Sep. 7, 2017, which is a National Stage ofPCT/JP2017/004385 filed Feb. 7, 2017, which claims the benefit ofpriority from Japanese Patent Application No. 2016-067323, filed Mar.30, 2016, the contents of each which is incorporated herewith byreference in its entirety.

TECHNICAL FIELD

The present disclosure relates to a control device, a control method,and a microscope device for operation.

BACKGROUND ART

Microscope devices have been used in surgical operations. A microscopedevice is configured such that an arm unit supports an electronicimaging microscope unit (a video microscope unit). An operator performsa surgical operation viewing an enlarged operating site using a videophotographed by the microscope unit.

With respect to such microscope devices, there have been demands forcontrol of positions and attitudes of microscope units thereof with highprecision to obtain desired videos. In particular, in a case in whichphotographing at a high magnification factor is being performed, aslight deviation of a position and an attitude of a microscope unitleads to a significant deviation of a video, and thus a position and anattitude of the microscope unit are required to be controlled with highprecision. A user normally moves a position and an attitude of such amicroscope unit using his or her hand; however, when highly precisepositioning is performed with his or her hand, the user has to dodelicate work, which increases a burden of the user and causes thepositioning work prolonged, and even leads to a lengthened operationtime.

Here, Patent Literature 1 discloses a technology relating to a scanningelectron microscope, rather than such a microscope device for operationdescribed above, for reducing a burden of manipulation of a user toobtain a desired image. Specifically, a stage on which a sample isplaced is automatically moved so that a desired image designated by auser is obtained in the technology disclosed in Patent Literature 1.According to this technology, a desired image can be automaticallyobtained only by performing a user's simple manipulation of designatingthe desired image, and thus a burden of the user can be reduced.

CITATION LIST Patent Literature

-   Patent Literature 1: JP 2012-138219A

DISCLOSURE OF INVENTION Technical Problem

Taking account of the above-described circumstance, a technology withrespect to a microscope device for operation that reduces a burden ofmanipulation of a user with respect to acquisition of a desired imageand improves user convenience as disclosed in Patent Literature 1 hasbeen demanded.

Therefore, the present disclosure proposes a novel and improved controldevice, control method, and microscope device for operation which canimprove user convenience.

Solution to Problem

According to the present disclosure, there is provided a control deviceincluding: a control unit configured to control a position and anattitude of a microscope unit by driving an arm unit that supports themicroscope unit on the basis of a captured image of an operating sitephotographed by the microscope unit during an operation so that aposition and attitude condition set before the operation is satisfied.The position and attitude condition is a condition that prescribes aposition and an attitude of the microscope unit with respect to theoperating site to obtain a desired captured image corresponding to theposition and attitude condition.

In addition, according to the present disclosure, there is provided acontrol method including: controlling, by a processor, a position and anattitude of a microscope unit by driving an arm unit that supports themicroscope unit on the basis of a captured image of an operating sitephotographed by the microscope unit during an operation so that aposition and attitude condition set before the operation is satisfied.The position and attitude condition is a condition that prescribes aposition and an attitude of the microscope unit with respect to theoperating site to obtain a desired captured image corresponding to theposition and attitude condition.

In addition, according to the present disclosure, there is provided Amicroscope device for operation including: a microscope unit configuredto photograph a captured image of an operating site; an arm unitconfigured to support the microscope unit; and a control deviceconfigured to control a position and an attitude of the microscope unitby driving the arm unit on the basis of a captured image of theoperating site photographed by the microscope unit during an operationso that a position and attitude condition set before the operation issatisfied. The position and attitude condition is a condition thatprescribes a position and an attitude of the microscope unit withrespect to the operating site to obtain a desired captured imagecorresponding to the position and attitude condition.

According to the present disclosure, a position and an attitude of amicroscope unit are controlled so that a position and an attitudecondition for obtaining a desired captured image set before an operationis satisfied on the basis of a captured image of an operating sitephotographed by the microscope unit during the operation. Thus, themicroscope unit can be automatically moved to a position and an attitudeat which the desired captured image is obtained without a complicatedmanipulation of a user. Therefore, a burden of the user can be reducedand user convenience can be improved.

Advantageous Effects of Invention

According to the present disclosure described above, user conveniencecan be enhanced. Note that the effects described above are notnecessarily limitative. With or in the place of the above effects, theremay be achieved any one of the effects described in this specificationor other effects that may be grasped from this specification.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating an example of a schematic configurationof a microscopic operation system according to a first embodiment.

FIG. 2 is a diagram illustrating a state of a operation in which themicroscopic operation system illustrated in FIG. 1 is being used.

FIG. 3 is a functional block diagram showing an example of a functionalconfiguration of a drive control system according to the firstembodiment.

FIG. 4 is a diagram illustrating an example of a GUI on which positionand attitude conditions are designated.

FIG. 5 is a diagram illustrating an example of a GUI on which positionand attitude conditions are registered.

FIG. 6 is a diagram illustrating an example of a GUI on which positionand attitude conditions are registered.

FIG. 7 is a diagram illustrating an example of a GUI on which positionand attitude conditions are registered.

FIG. 8 is a flowchart showing an example of a processing sequence of acontrol method according to the first embodiment.

FIG. 9 is a diagram for describing a search start position and a searchstart attitude.

FIG. 10 is a diagram for describing an eye position detection process onthe basis of a captured image.

FIG. 11 is a diagram for describing an eye position detection process onthe basis of a captured image.

FIG. 12 is a diagram for describing an eye position detection process onthe basis of a captured image.

FIG. 13 is a functional block diagram showing an example of a functionalconfiguration of a drive control system according to a secondembodiment.

FIG. 14 is a flowchart showing an example of a processing sequence of acontrol method according to the second embodiment.

FIG. 15 is a diagram for describing a modification of a position andattitude condition through learning.

FIG. 16 is a diagram illustrating an example of another GUI forregistering an instruction regarding an appearance of an image.

FIG. 17 is a diagram illustrating an example of another GUI forregistering an instruction regarding an appearance of an image.

FIG. 18 is a diagram illustrating an example of still another GUI forregistering an instruction of an appearance of an image.

FIG. 19 is a diagram illustrating an example of still another GUI forregistering an instruction of an appearance of an image.

FIG. 20 is a diagram illustrating an example of still another GUI forregistering an instruction of an appearance of an image.

MODE(S) FOR CARRYING OUT THE INVENTION

Hereinafter, (a) preferred embodiment(s) of the present disclosure willbe described in detail with reference to the appended drawings. In thisspecification and the appended drawings, structural elements that havesubstantially the same function and structure are denoted with the samereference numerals, and repeated explanation of these structuralelements is omitted.

Note that description will be provided in the following order.

1. First embodiment

1-1. Configuration of microscopic operation system

1-2. Configuration of drive control system

1-3. Processing sequence of control method

2. Second embodiment

2-1. Configuration of drive control system

2-2. Processing sequence of control method

3. Modified examples

3-1. Updating of position and attitude condition through learning

3-2. Other example of instruction included in position and attitudecondition

3-3. Other example of registration method of instruction regardingappearance of image

3-4. Other example of designation method for position and attitudecondition

3-5. Restriction on movement of microscope unit

4. Supplement

Note that, in the present specification, a “user” is assumed to mean atleast one of medical staff members (a doctor (an operator) who givestreatment on an operating site, an assistant, or the like) who use amicroscopic operation system and/or a drive control system which will bedescribed below. The “user” is described as an operator, an assistant,or the like particularly when he or she needs to be distinguished.

In addition, examples in which the technology according to the presentdisclosure is applied to an ophthalmic surgery will be described below.However, the present disclosure is not limited thereto and thetechnology according to the present disclosure may be applied to varioustypes of operations that can be performed using the microscopicoperation system that will be described below, for example, brainsurgeries, and the like.

1. First Embodiment

(1-1. Configuration of Microscopic Operation System)

A configuration of a microscopic operation system according to a firstembodiment of the present disclosure will be described with reference toFIG. 1. FIG. 1 is a diagram illustrating an example of a schematicconfiguration of the microscopic operation system according to the firstembodiment. Referring to FIG. 1, the microscopic operation system 3000is constituted by a microscope device 3100, a control device 3200, and adisplay device 3300.

The microscope device 3100 has a microscope unit 3110 for enlarging andobserving an observation object (an eye of a patient that is anoperating site), an arm unit 3120 that supports the microscope unit 3110at its leading end, and a base unit 3130 that supports a base end of thearm unit 3120.

The microscope unit 3110 is made up of an approximately cylindricalbarrel unit 3111, an imaging unit (not illustrated) provided inside thebarrel unit 3111, and an operating unit 3113 provided in a partialregion on the outer circumference of the barrel unit 3111. Themicroscope unit 3110 is an electronic imaging microscope unit (a videomicroscope unit) that images a captured image electronically with theimaging unit.

The aperture on the bottom end of the barrel unit 3111 is provided witha cover glass that protects the imaging unit inside. Light from anobservation target (hereinafter also called observation light) passesthrough the cover glass and is incident on the imaging unit inside thebarrel unit 3111. Note that a light source made up of a light-emittingdiode (LED) or the like, for example, may also be provided inside thebarrel unit 3111, and during imaging, light may be radiated from thelight source onto the observation target through the cover glass.

The imaging unit is made up of an optical system that condensesobservation light, and an image sensor that senses the observation lightcondensed by the optical system. The optical system is made up of acombination of multiple lenses, including a zoom lens and a focus lens,the optical characteristics of which are adjusted so that an image ofthe observation light is formed on the light-sensitive face of the imagesensor. The image sensor senses and photoelectrically converts theobservation light to thereby generate a signal corresponding to theobservation light, or in other words, an image signal corresponding tothe observed image. A sensor capable of color photography including aBayer array, for example, is used as the image sensor. The image sensormay be any of various known types of image sensors, such as acomplementary metal-oxide-semiconductor (CMOS) image sensor or acharge-coupled device (CCD) image sensor. The image signal generated bythe image sensor is transmitted to the control device 3200 as raw data.At this point, the transmission of the image signal may be conductedfavorably by optical communication. This is because at the surgeryvenue, an operator performs a operation while observing the state of alesion via the captured image, and thus for safer and more reliablesurgery, there is demand for the moving image of an eye that is theoperating site to be displayed as close to real-time as possible.Transmitting the image signal by optical communication makes it possibleto display the captured image without delay.

Note that the imaging unit also includes a drive mechanism that movesthe zoom lens and the focus lens of the optical system along the opticalaxis. By suitably moving the zoom lens and the focus lens with the drivemechanism, the magnification factor of the captured image and the focusdistance during imaging may be adjusted. Also, the imaging unit may beprovided with any of various types of functions typically provided inelectronic imaging microscope units, such as an auto exposure (AE)function, an auto focus (AF) function or the like.

In addition, the imaging unit may be configured as a so-called one-chipimaging unit that includes a single image sensor, or as a so-calledmulti-chip imaging unit that includes multiple image sensors. If theimaging unit has a multi-chip configuration, image signals correspondingto R, G, and B are generated by respective image sensors, for example,and a color image may be obtained by combining these image signals.Alternatively, the imaging unit may be configured to include a pair ofimage sensors for respectively acquiring image signals for the right eyeand the left eye corresponding to stereoscopic vision (3D display). Bypresenting 3D display, the operator is able to grasp the depth of theoperating site more accurately. Note that if the imaging unit has amulti-chip configuration, the optical system is provided with multiplesubsystems corresponding to each of the image sensors.

The operating unit 3113 is made up of elements such as a directionallever or switches, for example, and is an input unit that acceptsoperating input from a user. For example, via the operating unit 3113,the user is able to input an instruction to change the magnificationfactor of the observation target and the focus distance (focus). Byhaving the drive mechanism of the imaging unit suitably drive the zoomlens and the focus lens in accordance with the instruction, themagnification factor and the focus may be adjusted. As another example,via the operating unit 3113, the user is able to input an instruction totoggle the operating mode of the arm unit 3120 (a free mode and a lockedmode that will be described later). Note that when the user wants tomove the microscope unit 3110, it is anticipated that the user moves themicroscope unit 3110 while switching the operating mode of the arm unit3120 to the free mode by gripping and holding the barrel unit 3111.Consequently, the operating unit 3113 preferably is provided at aposition that allows easy operation with the fingers while the user isgripping the barrel unit 3111, to thereby allow the user to operate theoperating unit 3113 even while moving the barrel unit 3111.

The arm unit 3120 is configured as a result of multiple links (a firstlink 3123 a to a sixth link 3123 f) being rotatably joined to each otherby multiple joint units (a first joint unit 3121 a to a sixth joint unit3121 f).

The first joint unit 3121 a has an approximately cylindrical shape, andon the leading end (bottom end) thereof supports the top end of thebarrel unit 3111 of the microscope unit 3110, so as to allow rotationabout a rotation axis (first axis O₁) parallel to the central axis ofthe barrel unit 3111. Herein, the first joint unit 3121 a may beconfigured so that the first axis O₁ is aligned with the optical axis ofthe microscope unit 3110. Consequently, rotating the microscope unit3110 about the first axis O₁ makes it possible to change the field ofview as though rotating the captured image.

The first link 3123 a securely supports the first joint unit 3121 a onthe leading end thereof. Specifically, the first link 3123 a is anapproximately L-shaped rod-like member, the leading edge of whichextends in a direction orthogonal to the first axis O₁, while also beingconnected to the first joint unit 3121 a so that the end of that edgeabuts the top end on the outer circumference of the first joint unit3121 a. The second joint unit 3121 b is connected to the end of the baseedge of the approximate L-shape of the first link 3123 a.

The second joint unit 3121 b has an approximately cylindrical shape, andon the leading end thereof supports the base end of the first link 3123a, so as to allow rotation about a rotation axis (second axis O₂)orthogonal to the first axis O₁ The leading end of the second link 3123b is securely connected to the base end of the second joint unit 3121 b.

The second link 3123 b is an approximately L-shaped rod-like member, theleading edge of which extends in a direction orthogonal to the secondaxis O₂, while the end of that edge is securely connected to the baseend of the second joint unit 3121 b. The third joint unit 3121 c isconnected to the base edge of the approximate L-shape of the second link3123 b.

The third joint unit 3121 c has an approximately cylindrical shape, andon the leading end thereof supports the base end of the second link 3123b, so as to allow rotation about a rotation axis (third axis O₃)orthogonal to both the first axis O₁ and the second axis O₂. The leadingend of the third link 3123 c is securely connected to the base end ofthe third joint unit 3121 c. By rotating the configuration on theleading-end side, including the microscope unit 3110, about the secondaxis O₂ and the third axis O₃, the microscope unit 3110 may be moved tochange the position of the microscope unit 3110 on the horizontal plane.In other words, controlling the rotation about the second axis O₂ andthe third axis O₃ makes it possible to move the field of view of thecaptured image on a flat plane.

The third link 3123 c is configured to have an approximately cylindricalshape on the leading end side, and on the leading end of the cylindricalshape, the base end of the third joint unit 3121 c is securely connectedso that both have approximately the same central axis. The base end sideof the third link 3123 c has a rectangular column shape, and the fourthjoint unit 3121 d is connected to the end thereof.

The fourth joint unit 3121 d has an approximately cylindrical shape, andon the leading end thereof supports the base end of the third link 3123c, so as to allow rotation about a rotation axis (fourth axis O₄)orthogonal to the third axis O₃. The leading end of the fourth link 3123d is securely connected to the base end of the fourth joint unit 3121 d.

The fourth link 3123 d is a rod-like member that extends approximatelylinearly in a direction orthogonal to the fourth axis O₄, while alsobeing securely connected to the fourth joint unit 3121 d so that theleading end abuts the side face of the approximately cylindrical shapeof the fourth joint unit 3121 d. The fifth joint unit 3121 e isconnected to the base end of the fourth link 3123 d.

The fifth joint unit 3121 e has an approximately cylindrical shape, andon the leading end side thereof supports the base end of the fourth link3123 d, so as to allow rotation about a rotation axis (fifth axis O₅)parallel to the fourth axis O₄. The leading end of the fifth link 3123 eis securely connected to the base end of the fifth joint unit 3121 e.The fourth axis O₄ and the fifth axis O₅ are rotation axes enabling themicroscope unit 3110 to be moved in the vertical direction. By rotatingthe configuration on the leading-end side, including the microscope unit3110, about the fourth axis O₄ and the fifth axis O₅, the height of themicroscope unit 3110, or in other words the distance between themicroscope unit 3110 and the observation target, may be adjusted.

The fifth link 3123 e is made up of a combination of a first memberhaving an approximate L-shape with one edge extending in the verticaldirection while the other edge extends in the horizontal direction, anda rod-like second member that extends vertically downward from the partof the first member that extends in the horizontal direction. The baseend of the fifth joint unit 3121 e is securely connected near the topend of the part of the first member that extends in the verticaldirection of the fifth link 3123 e. The sixth joint unit 3121 f isconnected to the base end (bottom end) of the second member of the fifthlink 3123 e.

The sixth joint unit 3121 f has an approximately cylindrical shape, andon the leading end side thereof supports the base end of the fifth link3123 e, so as to allow rotation about a rotation axis (sixth axis O₆)parallel to the vertical direction. The leading end of the sixth link3123 f is securely connected to the base end of the sixth joint unit3121 f.

The sixth link 3123 f is a rod-like member that extends in the verticaldirection, with the base end securely connected to the top face of thebase unit 3130.

The allowable rotation range of the first joint unit 3121 a to the sixthjoint unit 3121 f is suitably set so that the microscope unit 3110 iscapable of making a desired motion. Consequently, in the arm unit 3120having the configuration described above, three degrees of translationalfreedom and three degrees of rotational freedom, for a total of sixdegrees of freedom, may be realized for the motion of the microscopeunit 3110. In this way, by configuring the arm unit 3120 so that sixdegrees of freedom are realized for the motion of the microscope unit3110, it becomes possible to freely control positions and attitudes ofthe microscope unit 3110 within the movable range of the arm unit 3120.Consequently, it becomes possible to observe an eye that is an operatingsite from any angle, and operations may be executed more smoothly.

Note that the configuration of the arm unit 3120 illustrated in thediagram is merely one example, and factors such as the number and theshapes (lengths) of the links constituting the arm unit 3120, as well asthe number and arrangement of the joint units and the directions of therotation axes may be designed suitably so that the desired degrees offreedom may be realized. For example, as described above, to move themicroscope unit 3110 freely, the arm unit 3120 preferably is configuredto have six degrees of freedom, but the arm unit 3120 may also beconfigured to have more degrees of freedom (in other words, redundantdegrees of freedom). When redundant degrees of freedom exist, in the armunit 3120, it becomes possible to change the attitude of the arm unit3120 while keeping the position and the attitude of the microscope unit3110 in a locked state. Consequently, control that is more convenient toan operator, such as control of the attitude of the arm unit 3120 sothat the arm unit 3120 does not interfere with the field of view of theoperator looking at the display device 3300, for example, may berealized.

Herein, the first joint unit 3121 a to the sixth joint unit 3121 f areprovided with actuators equipped with a drive mechanism such as a motor,an encoder that detects the rotation angle in each joint unit, and thelike. In addition, by having the control device 3200 suitable controldriving of each actuator provided for the first joint unit 3121 a to thesixth joint unit 3121 f, the attitude of the arm unit 3120, or in otherwords the position and the attitude of the microscope unit 3110, may becontrolled. Specifically, the control device 3200 is able to ascertainthe current attitude of the arm unit 3120 as well as the currentposition and attitude of the microscope unit 3110, on the basis ofinformation about the rotation angle of each joint unit detected by theencoder. The control device 3200 uses the ascertained information tocompute a control value for each joint unit (such as a rotation angle ora generated torque, for example) so that movement of the microscope unit3110 corresponding to operation input from the user is realized. Notethat at this point, the method by which the control device 3200 controlsthe arm unit 3120 is not limited, and any of various known controlmethods, such as force control or position control, may be applied.

For example, by having the operator perform suitable operation input viaan input device (not illustrated), the driving of the arm unit 3120 maybe suitably controlled by the control device 3200 in accordance with theoperation input, and the position and the attitude of the microscopeunit 3110 may be controlled. By such control, after moving themicroscope unit 3110 from an arbitrary position to an arbitraryposition, the microscope unit 3110 may be supported securely at a newposition. Note that with regard to the input device, in consideration ofthe operator's convenience, a device enabling operation even while theoperator is holding a surgical instrument in his or her hands, such as afootswitch, for example, is preferably applied. Also, non-contactoperation input may also be performed on the basis of gesture detectionor line-of-sight detection using wearable device or a camera providedinside the operating room. Consequently, even a user belonging to aclean area is able to operate equipment belonging to an unclean areawith a greater degree of freedom. Alternatively, the arm unit 3120 maybe operated by what is called a master-slave method. In this case, thearm unit 3120 may be operated remotely by a user via an input deviceinstalled at a location separate from the operating room.

Also, if force control is applied, what is called power-assist controlmay also be conducted, in which external force is received from a user,and the actuators of the first joint unit 3121 a to the sixth joint unit3121 f are driven so that the arm unit 3120 moves smoothly in responseto the external force. As a result, when the user grasps the microscopeunit 3110 to move the position directly, the microscope unit 3110 may bemoved with comparatively light force. Consequently, it becomes possibleto move the microscope unit 3110 more intuitively with a simplermanipulation, and user convenience may be improved.

In addition, the driving of the arm unit 3120 may be controlled so as toperform a pivot operation when the user manipulates the unit. Herein, apivot operation refers to an operation of moving the microscope unit3110 so that the optical axis of the microscope unit 3110 stays pointedat a certain point in a space (hereinafter called the pivot point). Apivot operation makes it possible to observe the same observationposition from various directions, thereby making more detailedobservation of the lesion possible. Note that if the microscope unit3110 is configured not to be able to adjust the focus, the pivotoperation is preferably performed in a state in which the distancebetween the microscope unit 3110 and the pivot point is fixed. In thiscase, it is sufficient to adjust the distance between the microscopeunit 3110 and the pivot point to the locked focus distance of themicroscope unit 3110. As a result, the microscope unit 3110 moves overthe face of a hemisphere centered on the pivot point and having a radiuscorresponding to the focus distance, and clear captured images areobtained even if the observation direction is changed. On the otherhand, if the microscope unit 3110 is configured to be able to adjust thefocus, the pivot operation may be performed with a variable distancebetween the microscope unit 3110 and the pivot point. In this case, forexample, the control device 3200 may calculate the distance between themicroscope unit 3110 and the pivot point on the basis of informationregarding rotation angles of the joint units detected by the encoder andautomatically adjust the focus of the microscope unit 3110 on the basisof the calculation result. Alternatively, in a case in which themicroscope unit 3110 has the AF function, the focus may be automaticallyadjusted through the AF function each time the distance between themicroscope unit 3110 and the pivot point changes due to a pivotoperation.

In addition, the first joint unit 3121 a to the sixth joint unit 3121 fmay also be provided with brakes that restrain rotation. The operationof such brakes may be controlled by the control device 3200. Forexample, in a case in which it is desirable to lock the position and theattitude of the microscope unit 3110, the control device 3200 appliesthe brake on each joint unit. As a result, the attitude of the arm unit3120, or in other words the position and the attitude of the microscopeunit 3110, may be locked without driving the actuators, and powerconsumption may be reduced. In a case in which it is desirable to movethe position and the attitude of the microscope unit 3110, it issufficient for the control device 3200 to release the brake on eachjoint unit and drive the actuators in accordance with a certain controlmethod.

Such a brake operation may be performed in response to operation inputperformed by a user via the operating unit 3113 described above. Whenthe user wants to move the position and the attitude of the microscopeunit 3110, the user operates the operating unit 3113 to release thebrake on each joint unit. As a result, the operating mode of the armunit 3120 switches to a mode allowing each joint unit to be rotatedfreely (free mode). Meanwhile, in a case in which the user wants to lockthe position and the attitude of the microscope unit 3110, the useroperates the operating unit 3113 to apply the brake on each joint unit.As a result, the operating mode of the arm unit 3120 switches to a modein which the rotation of each joint unit is restrained (locked mode).

The control device 3200 controls operations of the microscope device3100 and the display device 3300, and thereby controls overalloperations of the microscopic operation system 3000. For example, thecontrol device 3200 controls the driving of the arm unit 3120 by causingthe actuators of the first joint unit 3121 a to the sixth joint unit3121 f to operate in accordance with a certain control method. Asanother example, the control device 3200 changes the operating mode ofthe arm unit 3120 by controlling the operation of the brakes of thefirst joint unit 3121 a to the sixth joint unit 3121 f. As anotherexample, the control device 3200 performs various types of signalprocessing on an image signal acquired by the imaging unit of themicroscope unit 3110 in the microscope device 3100, and also makes theimage data to be displayed on the display device 3300. For the signalprocessing, any of various known types of signal processing, such as adevelopment process (demosaicing process), an image quality-improvingprocess (such as a band enhancement process, a super-resolution process,a noise reduction (NR) process, and/or a shake correction process),and/or an enlargement process (that is, a digital zoom process), may beperformed.

Note that the communication between the control device 3200 and themicroscope unit 3110, as well as the communication between the controldevice 3200 and the first joint unit 3121 a to the sixth joint unit 3121f, may be wired communication or wireless communication. In the case ofwired communication, communication using electrical signals may beconducted, or optical communication may be conducted. In this case, thetransmission cable used for wired communication may be configured as anelectrical signal cable, optical fiber, or a composite cable of the two,in accordance with the communication method. Meanwhile, in the case ofwireless communication, it is no longer necessary to lay down atransmission cable inside the operating room, and thus a situation inwhich the movement of medical staff inside the operating room is impededby such a transmission cable may be resolved.

The control device 3200 may be a processor such as a central processingunit (CPU) or a graphics processing unit (GPU), a control board on whicha processor and a storage element such as a memory are both mounted, orthe like. As a result of the processor of the control device 3200operating in accordance with a certain program, the various functionsdescribed above may be realized. Note that, in the example illustratedin the diagram, the control device 3200 is provided as a separate devicefrom the microscope device 3100, but the control device 3200 may also beunified with the microscope device 3100, such as by being installedinside the base unit 3130 of the microscope device 3100, for example.Alternatively, the control device 3200 may be made up of multipledevices. For example, by disposing a micro-computer, a control board orthe like in the microscope unit 3110 and each of the first joint unit3121 a to the sixth joint unit 3121 f of the arm unit 3120, andcommunicably connecting these control boards to each other, functionssimilar to the control device 3200 may be realized.

The display device 3300 displays an image corresponding to image datagenerated by the control device 3200 provided in an operating room undercontrol of the control device 3200. That is, the display device 3300displays an image of the eye that is an operating site photographed bythe microscope unit 3110 thereon. Note that the display device 3300 mayalso display various kinds of information regarding the operation, forexample, physical information of the patient or an operative procedure,instead of or along with the image of the eye. In this case, the displayof the display device 3300 may be appropriately switched through amanipulation of the user. Alternatively, a plurality of display devices3300 may be provided, and the plurality of display devices 3300 mayrespectively display the image of the eye and the various kinds ofinformation regarding the operation. Any of various known types ofdisplay devices, such as a liquid crystal display device or anelectroluminescence (EL) display device, for example, may be applied asthe display device 3300.

FIG. 2 is a diagram illustrating a state of an operation in which themicroscopic operation system 3000 illustrated in FIG. 1 is being used.FIG. 2 schematically illustrates the state in which an operator 3401 isperforming an operation with respect to a patient 3405 lying on apatient bed 3403, using the microscopic operation system 3000. Notethat, in FIG. 2, illustration of the control device 3200 is omitted fromthe configuration of the microscopic operation system 3000 andsimplified illustration of the microscope device 3100 is shown for thesake of simplicity.

As illustrated in FIG. 2, an enlarged image of an eye, which is anoperating site, photographed by the microscope device 3100 is displayedon the display device 3300 installed on a wall of an operating roomusing the microscopic operation system 3000 during an operation. Thedisplay device 3300 is installed at a position facing the operator 3401,and the operator 3401 performs various kinds of treatment on the eye,observing a state of the eye through the image projected on the displaydevice 3300.

(1-2. Configuration of Drive Control System)

A configuration of a drive control system according to the firstembodiment that is applied to the above-described microscopic operationsystem 3000 will be described with reference to FIG. 3. FIG. 3 is afunctional block diagram showing an example of a functionalconfiguration of the drive control system according to the firstembodiment.

Here, the drive control system according to the first embodiment is asystem that drives the arm unit 3120 of the microscope device 3100 ofthe microscopic operation system 3000 illustrated in FIG. 1 and controlsa position and an attitude of the microscope unit 3110 in order toacquire a captured image of an operating site that a user desires whenan operation starts. That is, the drive control system is a system thatautomatically moves the microscope unit 3110 to an initial position atwhich the desired captured image of the eye is likely to be obtainedwhen an operation starts. A series of control steps performed in thedrive control system according to the first embodiment to automaticallymove a position and an attitude of the microscope unit 3110 to theinitial position will also be referred to as initial operation controlbelow.

The initial operation control of the drive control system is started inaccordance with an instruction of a user. That is, an operation mode ofthe arm unit 3120 is in neither the above-described full mode nor thelocked mode, but is a so-called automatic operation mode while the drivecontrol system is activated and performs the initial operation control.Note that the automatic operation mode can appropriately start and stopin accordance with an instruction of a user.

Referring to FIG. 3, the drive control system 1 according to the firstembodiment has a pre-operation information acquisition unit 110, anin-operation information acquisition unit 120, a driving unit 130, and acontrol unit 140 for its functions.

The pre-operation information acquisition unit 110 is constituted byinput devices of various kinds (a touch panel, a remote controller, andthe like) provided in the microscopic operation system 3000. Thepre-operation information acquisition unit 110 acquires informationregarding position and attitude conditions that are conditions thatprescribe a position and an attitude of the microscope unit 3110 withrespect to an operating site to obtain a desired captured image of auser before the operation. Specifically, the position and attitudeconditions are set to at least include information with which a positionand an attitude of the microscope unit 3110 with respect to the eye,which is an operating site, can be uniquely determined. In the firstembodiment, the position and attitude conditions at least include aninstruction regarding an appearance of an image of the eye in thecaptured image (a position of the image of the eye in the capturedimage, a size of the image of the eye in the captured image, a vertexdirection in the captured image, or the like) and/or an instructionregarding a photographing direction of the eye (a positionalrelationship between an eye axis and an optical axis of the microscopeunit 3110). Here, in the first embodiment, a value determined by thedrive control system 1 is determined as a magnification factor of anoptical zoom and an electronic zoom of the microscope unit 3110. Thus,if the appearance of the image and the photographing direction aredesignated as position and attitude conditions, a position and anattitude of the microscope unit 3110 with respect to the eye forrealizing the conditions can be uniquely determined.

Note that the instructions included in the position and attitudeconditions may overlap each other. For example, in the above-describedexample, the appearance of the image may also include informationregarding photographing direction, such as whether the eye is beingviewed from vertically above or in a direction slightly oblique fromvertically above. Thus, in the first embodiment, levels of priority maybe set for the instructions included in the position and attitudeconditions. These levels of priority may be appropriately set by a userwhen the position and attitude conditions, which will be describedbelow, are registered. In a case in which the instructions included inthe position and attitude conditions overlap each other, a drive controlunit 14 of the control unit 140, which will be described below, controlsa position and an attitude of the microscope unit 3110 so that aninstruction having a higher level of priority is prioritized. Forexample, in a case in which a level of priority of the appearance of theimage is set to be higher than that of the photographing direction, aposition and an attitude of the microscope unit 3110 can be controlledsuch that a captured image sufficiently approximates an instructedappearance of the image while the photographing direction is set to asclose to an instructed direction (e.g., a vertically downward direction)as possible. Alternatively, in a case in which a level of priority ofthe photographing direction is set to be higher to certainly fulfill aninstruction regarding the photographing direction, a position and anattitude of the microscope unit 3110 can be controlled such that thecaptured image approximates an instructed appearance of the image asclosely as possible while realizing the photographing direction.

In the drive control system 1, for example, a plurality of sets ofdifferent position and attitude conditions are registered in a storageunit (not illustrated) provided in the drive control system 1 in advanceand a user designates his or her desired position and attitudeconditions among the sets of conditions, and thereby the pre-operationinformation acquisition unit 110 can acquire the position and attitudeconditions.

FIG. 4 illustrates an example of a graphical user interface (GUI) whenposition and attitude conditions are designated. FIG. 4 is a diagramillustrating an example of the GUI when position and attitude conditionsare designated. As illustrated in FIG. 4, for example, a plurality ofdifferent position and attitude conditions are displayed using icons 203on a display screen 201 constituting the pre-operation informationacquisition unit 110. Each of the icons 203 describes the name of anoperator and a location (an upper part or an ear-side part) of the eyeto be incised in the illustrated example. When the display screen 201 isintegrated with a touch panel and the user touches one of the icons 203,for example, a position and attitude condition corresponding to the icon203 can be designated. Note that control of displaying the display ofthe above-described GUI illustrated in FIG. 4 on the display screen 201constituting the pre-operation information acquisition unit 110 can beexecuted by the control unit 140.

In addition, when position and attitude conditions respectivelycorresponding to the icons 203 are to be registered in the storage unitin advance, for example, the user can perform the registration using aGUI illustrated in FIGS. 5 to 7. FIGS. 5 to 7 are diagrams illustratingan example of the GUI when position and attitude conditions areregistered. In FIGS. 5 to 7, the GUI for registering an instructionregarding an appearance of an image included in a position and attitudecondition is illustrated as an example.

For example, a circle 207 indicating a corneal ring portion and an arrow209 indicating a vertex direction of the patient are displayed on adisplay screen 205 in the GUI as illustrated in FIGS. 5 to 7. Aposition, a size, and a roundness of the displayed circle 207 can beappropriately modified in accordance with a manipulation of the user.The position of the circle 207 indicates a position of the corneal ringportion in a displayed captured image that has been actuallyphotographed by the microscope unit 3110 during an operation. The sizeof the circle 207 indicates a size of the corneal ring portion in thedisplayed captured image that has been actually photographed by themicroscope unit 3110 during the operation. In addition, the roundness ofthe circle 207 indicates a shape and a photographing direction of thecorneal ring portion in the displayed captured image that has beenactually photographed by the microscope unit 3110 during the operation.If the circle 207 is substantially a perfect circle as illustrated inFIGS. 5 and 7, for example, the roundness thereof indicates that thephotographing direction of the microscope unit 3110 is substantiallyvertically downward. In addition, if the circle 207 is an ellipse asillustrated in FIG. 6, the roundness thereof indicates that thephotographing direction of the microscope unit is a direction slightlyoblique with respect to the vertically downward direction. The userappropriately adjusts the position, the size, and the roundness of thecircle 207 to reproduce a captured image that he or she wants to viewduring the operation.

In addition, a direction of the displayed arrow 209 can be appropriatelymodified in accordance with a manipulation of the user. The direction ofthe arrow 209 indicates a vertex direction of the patient in thedisplayed captured image that has been actually captured by themicroscope unit 3110 during the operation. The user appropriatelyadjusts the direction of the arrow 209 to reproduce a captured imagethat he or she wants to view during the operation.

The display examples illustrated in FIGS. 5 to 7 show several examplesin which the circle 207 and the arrow 209 are disposed after the usercompletes adjustment. A position and attitude condition corresponding tothe display example illustrated in FIG. 5 can be, for example, aposition and attitude condition corresponding to an upward incision. Ina case in which position and attitude conditions are registered inaccordance with the display example illustrated in FIG. 5 and theinitial operation control is performed on the basis of these positionand attitude conditions, the microscope unit 3110 photographs the eye toposition the vertex part on a lower side of the imaging screen and thephotographing direction is substantially vertically downward.

In addition, a position and attitude condition corresponding to thedisplay example illustrated in FIG. 6 can be, for example, a positionand attitude condition corresponding to an upward incision, as in thedisplay example illustrated in FIG. 5. However, in a case in which theposition and attitude condition is registered in accordance with thedisplay example illustrated in FIG. 6 and the initial operation controlis performed on the basis of the position and attitude condition, theeye is photographed to position the vertex part on the lower side of theimaging screen and the photographing direction is a direction slightlyoblique with respect to the vertically downward direction.

In addition, a position and attitude condition corresponding to thedisplay example illustrated in FIG. 7 can be, for example, a positionand attitude condition corresponding to an incision to an ear side. In acase in which the position and attitude condition is registered inaccordance with the display example illustrated in FIG. 7 and theinitial operation control is performed on the basis of the position andattitude condition, the eye is photographed to position the vertex parton a right side of the imaging screen and the photographing direction issubstantially vertically downward.

The state of the circle 207 and the arrow 209 after the user completesadjustment are registered in the storage unit as an instructionregarding an appearance of an image included in the position andattitude condition. Since the user can intuitively register theappearance of the image that he or she wants to obtain during theoperation using the GUI, the position and attitude condition can beregistered more simply. Note that the control of displaying theabove-described display using the GUI illustrated in FIGS. 5 to 7 on thedisplay screen 205 can be executed by the control unit 140. In addition,the display screen 205 may be a display screen of the display deviceconstituting the pre-operation information acquisition unit 110, or adisplay screen of a separate display device provided in the drivecontrol system 1.

The pre-operation information acquisition unit 110 provides informationregarding the position and attitude condition acquired before theoperation to a position and attitude condition setting unit 141 of thecontrol unit 140 which will be described below.

The in-operation information acquisition unit 120 acquires various kindsof information necessary for controlling a position and an attitude ofthe microscope unit 3110 during the operation (which will also bereferred to as in-operation information below). The in-operationinformation may be information indicating a positional relationshipbetween the eye that is the operating site and the microscope unit 3110.The in-operation information acquisition unit 120 is configured by themicroscope unit 3110, and at least acquires information regarding acaptured image (captured image information) as the in-operationinformation. In addition, the microscopic operation system 3000 may alsoinclude a marker (e.g., a magnetic marker) provided in the vertexdirection of the patient bed 3403 and a sensor (e.g., a magnetic sensor)that detects the marker, and the in-operation information acquisitionunit 120 may be configured to include the marker and the sensor. In thatcase, the in-operation information acquisition unit 120 can acquireinformation regarding a relative position corresponding to the head partof the patient bed 3403 with respect to the microscope unit 3110 on thebasis of a detection value of the sensor as the in-operationinformation.

The in-operation information acquisition unit 120 acquires thein-operation information when necessary during an operation. Note that,according to the above-described example, acquisition of captured imageinformation and acquisition of information regarding a relative positionof the patient bed 3403 may be performed together at all times, orappropriately switched between in a time sequence so that only one kindof information is acquired.

The in-operation information acquisition unit 120 provides the acquiredin-operation information to a state recognition unit 142 of the controlunit 140 which will be described below.

The driving unit 130 is constituted by an actuator provided in eachjoint unit of the arm unit 3120 of the microscope device 3100. Thedriving unit 130 is driven under control of a drive control unit 144 ofthe control unit 140, which will be described below, so that a positionand attitude condition is satisfied on the basis of a captured imageacquired during the operation. Accordingly, the arm unit 3120 is drivenand a position and an attitude of the microscope unit 3110 can becontrolled so that a desired captured image that satisfies the positionand attitude condition is obtained.

The control unit 140 is configured by the control device 3200, andcomprehensively controls processes performed in the drive control system1. The control unit 140 has the position and attitude condition settingunit 141, the state recognition unit 142, a state comparison unit 143,and the drive control unit 144 for its functions. These functions can berealized through operations of a processor included in the control unit140 in accordance with a predetermined program.

The position and attitude condition setting unit 141 sets the positionand attitude condition on the basis of information regarding theposition and attitude condition acquired before the operation providedfrom the pre-operation information acquisition unit 110. Specifically,the position and attitude condition setting unit 141 extracts a featureamount of a desired captured image corresponding to the position andattitude condition from the information regarding the position andattitude condition and stores a parameter indicating the feature amount.The parameter is, for example, a center position of the ellipse or thecircle corresponding to the corneal ring portion in the captured image,a long diameter and a short diameter of the ellipse or the circlecorresponding to the corneal ring portion in the captured image, a longaxis direction of the ellipse or the circle corresponding to the cornealring portion in the captured image, a vertex direction in the capturedimage, or the like.

The position and attitude condition setting unit 141 provides theinformation regarding the set position and attitude condition (i.e.,information regarding the extracted feature amount (the parameter) tothe state comparison unit 143 and the drive control unit 144.

The state recognition unit 142 recognizes a state of a current capturedimage acquired by the microscope unit 3110 on the basis of thein-operation information provided from the in-operation informationacquisition unit 120. The state of the captured image can be informationcorresponding to the position and attitude condition included in thecaptured image, i.e., an appearance of an image of the eye included inthe captured image, and a photographing direction in the captured image.The state recognition unit 142 recognizes the state on the basis of thecaptured image information provided from the in-operation informationacquisition unit 120 using any of various kinds of image recognitiontechnology. At this time, in a case in which the in-operationinformation provided from the in-operation information acquisition unit120 includes information regarding a relative position of the patientbed 3403 based on a detection value of a magnetic sensor or the like,the state recognition unit 142 may recognize a rough position of the eyeand the vertex direction in the captured image on the basis ofinformation regarding the position of the patient bed 3403.

The state recognition unit 142 provides information regarding therecognized state of the current captured image to the state comparisonunit 143 and the drive control unit 144.

The state comparison unit 143 compares the position and attitudecondition set by the user before the operation and the state of thecurrent captured image recognized by the state recognition unit 142, anddetermines whether the state of the current captured image approximatesthe state of the desired captured image corresponding to the positionand attitude condition. Specifically, the state comparison unit 143extracts a feature amount of the current captured image on the basis ofthe information regarding the state of the current captured imageprovided by the state recognition unit 142, similar to the extractedfeature amount of the desired captured image corresponding to theabove-described position and attitude condition. Then, whether the stateof the current captured image approximates the state of the desiredcaptured image corresponding to the position and attitude condition isdetermined by comparing the feature amounts of the images with eachother.

In a case in which the states of both images are determined to bedistant from each other, the state comparison unit 143 issues aninstruction to the drive control unit 144 to drive the arm unit 3120 sothat the desired captured image corresponding to the position andattitude condition is obtained. On the other hand, in a case in whichthe states of both images are determined to approximate each other, thedetermination indicates that an image that sufficiently approximates thedesired captured image has already been captured, and thus the statecomparison unit 143 ends the process without issuing any particularinstruction to the drive control unit 144.

In a case in which an instruction has been received from the statecomparison unit 143, the drive control unit 144 drives the driving unit130 on the basis of the information regarding the position and attitudecondition set before the operation provided from the position andattitude condition setting unit 141 and the information regarding thestate of the current captured image provided from the state recognitionunit 142 so that the position and attitude condition is satisfied, andthereby the position and the attitude of the microscope unit 3110 areupdated. The in-operation information acquisition unit 120 acquiresin-operation information again with respect to the updated new positionand attitude, and a state recognition process and a state comparisonprocess are performed by the state recognition unit 142 and the statecomparison unit 143 respectively again on the basis of the in-operationinformation.

In addition, the drive control unit 144 may drive the driving unit 130in accordance with an instruction given from outside by a user to changethe position and the attitude of the microscope unit 3110. For example,when a series of processes relating to the above-described initialoperation control is started, the drive control unit 144 may drive thedriving unit 130 in accordance with the instruction from the user tomove the position and the attitude of the microscope unit 3110 to aninitial position (search start position) and an initial attitude (searchstart attitude) at which the series of processes relating to the initialoperation control is started.

The configuration of the drive control system 1 according to the firstembodiment has been described above. Note that the function of thecontrol unit 140 will be described again in more detail with referenceto FIG. 8.

(1-3. Processing Sequence of Control Method)

A processing sequence of a control method according to the firstembodiment will be described with reference to FIG. 8. FIG. 8 is aflowchart showing an example of the processing sequence of the controlmethod according to the first embodiment. Note that the processes shownin FIG. 8 correspond to processes executed by the control unit 140 ofthe above-described drive control system 1 illustrated in FIG. 3.

Referring to FIG. 8, first, a position and attitude condition is set inaccordance with an input of a user before an operation (Step S101) inthe control method according to the first embodiment. The process ofStep S101 corresponds to the process executed by the position andattitude condition setting unit 141 illustrated in FIG. 3. Note that,since the position and attitude condition has been described above indetail, description thereof is omitted here.

Next, the microscope unit 3110 is moved to take a search start positionand a search start attitude in accordance with an instruction of theuser on a start of initial operation control (Step S103). The process ofStep S103 corresponds to the process executed by the drive control unit144 illustrated in FIG. 3.

FIG. 9 is a diagram for describing a search start position and a searchstart attitude. In FIG. 9, simplified illustration of the microscopedevice 3100 illustrated in FIG. 1 is shown for the sake of simplicity.

FIG. 9(a) shows a position and an attitude of the microscope unit 3110before the drive control system 1 starts the initial operation control,i.e., at the time of receipt or delivery. In the first embodiment, theposition and the attitude of the microscope unit 3110 before the initialoperation control is started may not be fixed.

The microscope unit 3110 is moved to the search start position and thesearch start attitude from the above-described state in accordance withan instruction of the user that the initial operation control be started(FIG. 9(b)). The search start position and the search start attitude area position and an attitude in which the microscope unit is placedimmediately above and relatively higher than the patient bed 3403 andthe optical axis is oriented in a vertically downward direction, asillustrated in FIG. 9(b). Normally in ophthalmic surgery, the patient3405 lies on his or her side face up on the patient bed 3403 and theoperation is performed on the patient having his or her eye oriented ina vertically upward direction in most cases (see FIG. 2), and thus acaptured image overlooking a range including the eye can be obtained bysetting the search start position, and the search start attitude asdescribed above. Here, a prescribed value of magnification factor of themicroscope unit 3110 is stipulated in the first embodiment as describedabove. However, after the microscope unit 3110 is moved to the searchstart position and the search start attitude in Step S103, an eyedetection process can be performed in a process of ascertaining a stateof a current captured image (Step S105) as will be described below.Thus, when the microscope unit 3110 is moved to the search startposition and the search start attitude, a magnification factor of themicroscope unit 3110 may be automatically set to have as wide an angleas possible, regardless of the stipulated value. Accordingly, alikelihood that an eye will be included in the captured image at thesearch start position and the search start attitude increases, and thusthe eye detection process can be performed more efficiently. Note that,when a process of updating the position and the attitude of themicroscope unit 3110 is performed in Step S109 in this case, as will bedescribed below, the magnification factor of the microscope unit 3110may be adjusted to the stipulated value at an appropriate timing.

Note that the above-described search start position and search startattitude are merely examples, and the search start position and thesearch start attitude may be appropriately set by the user in accordancewith an operative procedure in the first embodiment.

In addition, the drive control unit 144 may control the position and theattitude of the microscope unit 3110 before the start of the initialoperation control so that the position and the attitude of themicroscope unit 3110 at the time of receipt or delivery become the sameas the search start position and the search start attitude. In thiscase, the process of Step S103 can be appropriately omitted.

Furthermore, the search start position and the search start attitude maynot be set in advance, and the user may perform a manual manipulation tomove the microscope unit 3110 to an arbitrary position and attitude atwhich the eye is likely to be included in the captured image, instead ofthe process of Step S103.

When the microscope unit 3110 is moved to the search start position andthe search start attitude in Step S101, a state of a current capturedimage is recognized on the basis of the in-operation information (StepS105). The process of Step S105 corresponds to the process executed bythe state recognition unit 142 illustrated in FIG. 3. Note that, sincethe in-operation information has been described above in detail,description thereof is omitted here.

Specifically, a position of the eye is first detected from capturedimage information included in the in-operation information in Step S105.Any of various known image recognition technologies may be used in theprocess.

FIGS. 10 to 12 are diagrams for describing a process of detecting aposition of the eye on the basis of a captured image. FIG. 10schematically illustrates a state of the patient 3405 lying on thepatient bed 3403 viewed from above during ophthalmic surgery. A drape211 covers the entire body of the patient 3405 who is lying on his orher back on the patient bed 3403 during the ophthalmic surgery asillustrated in FIG. 10. The drape 211 has an opening at a positioncorresponding to the eye 213 that is the operating site to expose onlythe eye 213 therethrough. An instrument 215 keeps the eye 213 openduring the operation.

FIG. 11 illustrates an example of a captured image captured by themicroscope unit 3110 at a search start position and a search startattitude. As illustrated in FIG. 11, for example, an image overlookingthe head of the patient and its vicinity from above is obtained at thesearch start position and the search start attitude. The eye 213 can bedetected in the captured image by detecting, for example, the white ofthe eye from the color of the drape 211. Note that, in a case in whichthe in-operation information includes information regarding a relativeposition of the patient bed 3403 on the basis of a detection value of amagnetic sensor or the like, a rough position of the eye 213 may bedetected on the basis of the information.

When the position of the eye 213 is detected in Step S105, then acorneal ring portion and a vertex direction are detected in the image ofthe eye 213 included in the captured image. FIG. 12 illustrates anexample of the image of the eye 213 included in the captured imagecaptured by the microscope unit 3110. As illustrated in FIG. 12, acorneal ring portion 216 is the boundary between the white of the eyeand the iris 217, and thus the corneal ring portion 216 can be detectedby detecting a change in color in the image of the eye 213 andspecifying the boundary on which the color changes.

In addition, the vertex direction can be detected by recording patternsof blood vessels and the iris 217 of the eye 213 of the patient 3405included in a captured image of the eye 213 captured at a sittingposition in advance and comparing the pre-recorded patterns withpatterns thereof that are recognized in an image of the eye 213 includedin another captured image captured by the microscope unit 3110.Alternatively, in a case in which a captured image that is likely toinclude a positional relationship between the drape 211 and the eye 213as illustrated in FIG. 11 is obtained, a direction of the body of thepatient 3405 may be identified from the positional relationship and thenthe vertex direction may be detected on the basis of the direction ofthe body. Further alternatively, in a case in which the in-operationinformation includes information regarding a relative position of thehead of the patient bed 3403 based on a detection value of a magneticsensor or the like, the vertex direction may be detected on the basis ofthis information.

When the state of the current captured image is recognized in Step S105,it is then determined whether the current captured image approximates adesired captured image corresponding to a position and attitudecondition set before the operation (Step S107). The process of Step S107corresponds to the process executed by the state comparison unit 143illustrated in FIG. 3.

Specifically, in Step S107, a feature amount of the current capturedimage is extracted from the state of the captured image recognized inStep S105. The feature amount is extracted as a parameter similar to aparameter indicating a feature amount of the desired captured imagecorresponding to the above-described position and attitude condition(e.g., a center position of an ellipse or a circle corresponding to thecorneal ring portion of the captured image, lengths of a long diameterand a short diameter of the ellipse or the circle corresponding to thecorneal ring portion of the captured image, a long axis direction of theellipse or the circle corresponding to the corneal ring portion of thecaptured image, the vertex direction in the captured image, etc.). Then,using an appropriate evaluation function with the parameters asvariables, degrees of matching with respect to instructed detailsincluded in the position and attitude condition (specifically, a degreeof matching of an appearance of an image under the position and attitudecondition set before the operation with an appearance of the currentcaptured image, and a degree of matching of a photographing directionunder the position and attitude condition set before the operation witha photographing direction in the current captured image) are calculatedas distance indicators.

In a case in which all of the distance indicators are smaller than apredetermined threshold value, the state of the current captured imageis determined to sufficiently approximate the state of the desiredcaptured image corresponding to the position and attitude condition.Since this case indicates that an image that sufficiently approximatesthe desired captured image has already been captured, the series ofprocess ends. An example of the position and the attitude of themicroscope unit 3110 at the time point at which the series of processesends (i.e., the time point at which the initial operation control ends)is illustrated in FIG. 9(c).

On the other hand, in a case in which any one of the calculated rangeindices is greater than the predetermined threshold value, the state ofthe current captured image is determined to be distant from the state ofthe desired captured image corresponding to the position and attitudecondition. In that case, the process proceeds to Step S109.

Note that levels of priority can be included in an instruction includedin the position and attitude condition as described above. In the casein which levels of priority are included in an instruction included inthe position and attitude condition, the threshold value with respect tothe above-described range indices may be changed in accordance with thelevels of priority for each instruction. For example, in a case in whicha level of priority of an appearance of an image is set to be higherthan that of a photographing direction, a threshold value with respectto a distance indicator indicating a degree of coincidence of theappearance of the image may be lower than a threshold value with respectto a distance indicator indicting a degree of coincidence of thephotographing direction.

In Step S109, the arm unit 3120 is driven so that the position andattitude condition set before the operation is satisfied, and thus theposition and the attitude of the microscope unit 3110 are updated. Then,returning to Step S105, the processes of Step S105 and Step S107 arerepeated in the updated new position and attitude. Note that the processof Step S109 corresponds to the process executed by the drive controlunit 144 illustrated in FIG. 3.

Specifically in Step S109, specific details of the updating (i.e., anamount of movement of the microscope unit 3110) can be set on the basisof the position and attitude of the microscope unit 3110 at the timepoint and a state of a captured image acquired at the time point. Forexample, in a case in which the microscope unit 3110 is in a positionand an attitude that approximate the search start position and thesearch start attitude, the position of the microscope unit 3110 is movedto position the eye 213 at the center of the captured image due to themovement in the horizontal direction with the height and the attitude ofthe microscope unit 3110 maintained. In addition, in a case in which theeye 213 reaches the center of the captured image and the magnificationfactor of the microscope unit 3110 is changed to have a wide angle atthe search start position and the search start attitude, for example,the magnification factor is changed such that that of the microscopeunit 3110 has the value stipulated for the drive control system 1.Further, in a case in which the above-described task is achieved, forexample, the microscope unit 3110 approaches the eye 213 until the sizeof the eye 213 in the captured image approximates a size of the eye (asize of the corneal ring portion) in the appearance of the imageincluded in the position and attitude condition set before theoperation. Further, in a case in which the above-described task isachieved, for example, the position and the attitude of the microscopeunit 3110 are finely adjusted so as to approximate the appearance of theimage and the photographing direction included in the position andattitude condition set before the operation.

However, a time is taken to move the microscope unit 3110 to the finalposition and attitude in the above-described phased movement. Thus,actually, the final position and attitude, a movement route to the finalposition and attitude, a movement time taken to reach the final positionand attitude, and the like are predicted and the position and attitudeof the microscope unit 3110 may be updated to reduce the movement timeto the shortest time on the basis of the prediction result.

The processing sequence of the control method according to the firstembodiment has been described above.

Here, a position and an attitude of a microscope unit are in generaladjusted by a user using his or her hand in a operation in which amicroscope device is used so far. For example, the user grabs themicroscope unit by himself or herself and moves the microscope unit toan approximate position, then finely adjusts the position of themicroscope unit using a foot pedal powered by electricity, and therebyadjusts a position and an attitude of the microscope. Complicated workthat takes a fair time is necessary to adjust the position and theattitude of the microscope unit, which imposes a burden on the user asdescribed above. In particular, in a case in which photographing isperformed with a high magnification factor, a significant change isshown in a captured image only by slightly changing a position and anattitude of the microscope unit, and thus adjustment for obtaining adesired captured image needs to be more delicate, and thus a work timeand a burden of the user further increases. An increase in the work timemeans a lengthened operation time, which also increases a burden of apatient.

With regard to the above-described matter, the microscope unit 3110 canbe automatically moved to a position and an attitude at which thedesired captured image is obtained when the operation starts, asdescribed above according to the first embodiment. Thus, the user canobtain the desired captured image more easily without spending time andefforts in adjustment. Therefore, a shortened operation time and areduced burden of the user and the patient can be realized.

2. Second Embodiment

A second embodiment of the present disclosure will be described. In theinitial operation control according to the above-described firstembodiment, in the case in which the state of the current captured imageobtained by the microscope unit 3110 is determined to approximate thestate of the desired captured image corresponding to the position andattitude condition set before the operation, the series of processesperformed in the drive control system 1 ends. That is, in the initialoperation control according to the first embodiment, driving control ofthe microscope unit 3110 by the drive control system 1 ends at the timepoint at which the position and the attitude of the microscope unit 3110are controlled so that the set position and attitude condition aresatisfied. Meanwhile, the eye may move due to autokinesis of the patient3405 or treatment of the operator 3401 during the operation. In such acase, it is desirable to continuously control the position and attitudeof the microscope unit 3110 so that the position and attitude conditionis further satisfied even when the eye has been moved and the movementof the eye is traced, depending on an operative procedure or details ofthe position and attitude condition.

Thus, in the second embodiment, a drive control system is configuredsuch that a position and an attitude of the microscope unit 3110 atwhich a position and attitude condition is likely to be satisfied arecontinuously controlled until a user issues an end instruction, ratherthan ending the control at a time point at which the position andattitude condition is satisfied once. Accordingly, in the secondembodiment, desired captured images corresponding to the position andattitude condition are continuously obtained even when the eye is moved,and thus user convenience can be improved. Note that control of thedrive control system according to the second embodiment in which aposition and an attitude of the microscope unit 3110 are automaticallymoved so that the position and attitude condition is satisfied whilemovement of the eye is traced until an end instruction is input willalso be referred to as trace operation control.

Note that the drive control system according to the second embodimentexecutes processes similar to those in the first embodiment except thata trigger for ending control is different. Thus, differences of thesecond embodiment from the first embodiment will be mainly describedbelow, and detailed description of overlapping matters with the firstembodiment will be omitted.

(2-1. Configuration of Drive Control System)

A configuration of the drive control system according to the secondembodiment applied to the above-described microscopic operation system3000 will be described with reference to FIG. 13. FIG. 13 is afunctional block diagram showing an example of a functionalconfiguration of the drive control system according to the secondembodiment.

Referring to FIG. 13, the drive control system 2 according to the secondembodiment has the pre-operation information acquisition unit 110, thein-operation information acquisition unit 120, the driving unit 130, anda control unit 150 for its functions. Note that, since functions of thepre-operation information acquisition unit 110, the in-operationinformation acquisition unit 120, and the driving unit 130 are similarto those of the first embodiment, detailed description thereof will beomitted here.

The control unit 150 has the position and attitude condition settingunit 141, the state recognition unit 142, an end determination unit 153,and the drive control unit 144 for its functions. These functions can berealized by a processor constituting the control unit 150 that operatesin accordance with a predetermined program. Note that, since thefunctions of the position and attitude condition setting unit 141, thestate recognition unit 142, and the drive control unit 144 are similarto those of the first embodiment, detailed description will be omittedhere.

The end determination unit 153 determines whether there is aninstruction issued from a user to end the trace operation control. In acase in which there is no end instruction, the end determination unit153 issues an instruction to drive the arm unit 3120 to the drivecontrol unit 144 so that a desired captured image corresponding to aposition and attitude condition is obtained. The drive control unit 144drives the driving unit 130 complying with the instruction so that theposition and attitude condition set before an operation is satisfied,and thereby a position and an attitude of the microscope unit 3110 areupdated. The in-operation information acquisition unit 120 acquiresin-operation information at the updated new position and attitude andthe state recognition unit 142 performs a state recognition processagain on the basis of the in-operation information. On the other hand,in a case in which it is determined that there is an end instruction,the process ends without a particular instruction issued by the enddetermination unit 153 to the drive control unit 144.

The configuration of the drive control system 2 according to the secondembodiment has been described above.

(2-2. Processing Sequence of Control Method)

A processing sequence of a control method according to the secondembodiment will be described with reference to FIG. 14. FIG. 14 is aflowchart showing an example of the processing sequence of the controlmethod according to the second embodiment. Note that the processes shownin FIG. 14 correspond to processes executed by the control unit 150 ofthe drive control system 2 according to the second embodiment shown inFIG. 13.

Referring to FIG. 14, similar processes to those of Step S101 to StepS105 of the control method according to the first embodiment areexecuted in Step S201 to Step S205 of the control method according tothe second embodiment. That is, a position and attitude condition is setbefore an operation (Step S201), the microscope unit 3110 is moved tohave a search start position and a search start attitude (Step S203),and a state of a current captured image is recognized on the basis ofin-operation information (Step S205).

In the control method according to the second embodiment, it is thendetermined whether the user has input an end instruction, unlike in thefirst embodiment (Step S207). In a case in which an end instruction isdetermined to have been input in Step S207, a series of processes end.On the other hand, in a case in which no end instruction is determinedto have been input in Step S207, the process proceeds to Step S209. Notethat the process of Step S207 corresponds to the process executed by theend determination unit 153 illustrated in FIG. 13.

In Step S209, a similar process to that of Step S109 of the controlmethod according to the first embodiment is executed. That is, the armunit 3120 is driven so that the position and attitude condition setbefore the operation is satisfied, and thus the position and theattitude of the microscope unit 3110 are updated. Then, returning toStep S205, the processes of Step S205 and Step S207 are repeated in theupdated new position and attitude.

The processing sequence of the control method according to the secondembodiment has been described. Also in the second embodiment, themicroscope unit 3110 can be automatically moved to a position and anattitude obtained in desired captured image, as in the first embodiment.Thus, similar effect to that of the first embodiment, i.e., a shortenedoperation time and a reduced burden of the user and the patient can berealized. Further, according to the second embodiment, even in a case inwhich an eye moves during an operation, the microscope unit 3110 isautomatically moved so that the position and attitude condition issatisfied while tracing the movement of the eye. Thus, a desiredcaptured image can be obtained at all times regardless of movement ofthe eye, and thus user convenience can be improved.

Note that, although the process of moving the microscope unit 3110 tohave the search start position and the search start attitude is executedin Step S203 in the example shown in FIG. 14, the second embodiment isnot limited thereto. For example, a situation in which the traceoperation control according to the second embodiment is executedconsecutively after the initial operation control according to the firstembodiment ends is also assumed. In this case, a position and anattitude of the microscope unit 3110 might have been controlled suchthat a desired captured image is substantially obtained at the positionand the attitude at the time point at which the trace operation controlis about to be executed. If the process of Step S203 is executed in thatstate, the position and the attitude of the microscope unit 3110 arereset, which is inefficient. Thus, the process of Step S203 may beomitted in the case in which the microscope unit 3110 is expected tohave already obtained an image that is close to the desired capturedimage, as in the case in which the trace operation control is executedconsecutively after the initial operation control.

In addition, although the position and the attitude of the microscopeunit 3110 can be controlled in accordance with movement of the eye whennecessary in the second embodiment, there is concern that, if themicroscope unit 3110 is moved even in response to, for example, trivialmovement of the eye or regular movement of the eye with very shortintervals, unstable captured images are displayed, which ratherdeteriorates convenience of an operator. The position and the attitudeof the microscope unit 3110 may be controlled so as not to tracemovement of the eye when the movement of the eye is considered to betrivial. In addition, in a case in which the eye regularly moves withshort intervals, an average pattern of movement of the eye within agiven time may be taken and the position and the attitude of themicroscope unit 3110 may be controlled so as to trace the averagepattern. That is, the position and the attitude of the microscope unit3110 may be controlled so as to trace only extensive movement of theeye.

3. Modified Examples

Several modified examples of the above-described first and secondembodiments will be described.

(3-1. Updating of Position and Attitude Condition Through Learning)

In the description above, position and attitude conditions areregistered in advance in the drive control systems 1 and 2 and one isappropriately selected therefrom when the initial operation control andthe trace operation control are performed. However, in that scheme inwhich the position and attitude conditions are registered in advance, itcan be assumed that a difference exists between a captured imagecorresponding to a position and attitude condition that has already beenregistered and an image that a user actually wants to view as a resultthat an operation is actually performed with the initial operationcontrol or the trace operation control. In this case, after themicroscope unit 3110 is moved so that the registered position andattitude condition registered using the initial operation control andthe trace operation control is satisfied, an operator has to manuallymodify the position and the attitude of the microscope unit 3110 toobtain his or her really desired captured image.

In the first and second embodiments, in the case in which an operatormanually modifies a position and an attitude of the microscope unit 3110as described above, a registered position and attitude condition may beupdated on the basis of the modification result. That is, the controlunits 140 and 150 of the drive control systems 1 and 2 may have afunction of updating a registered position and attitude conditionthrough learning.

Here, a case in which an appearance of an image included in a positionand attitude condition is updated through learning will be described asan example. FIG. 15 is a diagram for describing updating of a positionand attitude condition through learning. In FIG. 15, the appearance ofthe image included in the position and attitude condition is illustratedin a similar form to the GUI for registering the appearance of the imagedescribed with reference to FIGS. 5 to 7 for the sake of convenience.That is, a circle 207 indicates a corneal ring portion and an arrow 209indicates a vertex direction in FIG. 15.

FIG. 15 (a) illustrates an example of the appearance of the imageincluded in the registered position and attitude condition beforelearning is performed. Before learning is performed, for example, it isassumed that the appearance of the image, in which the corneal ringportion is positioned substantially at the center of a captured image asillustrated in FIG. 15 (a), is registered as a position and attitudecondition.

On the other hand, it is assumed that the initial operation control orthe trace operation control is performed so that the position andattitude condition including the appearance of the image illustrated inFIG. 15 (a) is satisfied and then the operator modifies the position andthe attitude of the microscope unit 3110. The appearance of the imagecorresponding to the modified captured image is illustrated in FIG. 15(b). According to the illustrated example, the corneal ring portion ispresent at a position slightly deviating to the left from the center ofthe captured image. Note that the appearance of the image correspondingto the modified captured image can be specified by performing any ofvarious kinds of image recognition processing on the captured image.

In the case in which the operator performs a modification after theinitial operation control or the trace operation control as describedabove, the appearance of the modified image is considered as anappropriate appearance of the image for the operator. Thus, the controlunits 140 and 150 of the drive control systems 1 and 2 updatesinstruction details regarding the appearance of the image included inthe position and attitude condition set this time with details of theappearance of the modified image. Accordingly, in a case in which theoperator designates the position and attitude information and performsthe initial operation control or the trace operation control next time,the position and the attitude of the microscope unit 3110 are controlledso as to realize the appearance of the modified image, and thusconvenience of the operator can be improved.

Note that learning may be performed practically on the basis of resultsof a plurality of modifications, rather than a result of onemodification. In this case, the control units 140 and 150 may obtain anappearance of a modified average image from appearances of a pluralityof images corresponding to modified captured images, which are obtainedas a result of the plurality of modifications, and perform updating witha position and attitude condition registered on the basis of theappearance of the average image.

(3-2. Other Example of Instruction Included in Position and AttitudeCondition)

An appearance of an image and a photographing direction are exemplifiedas instructions included in a position and attitude condition in theabove description. However, the first and second embodiments are notlimited thereto. A position and attitude condition may only include oneof the instructions regarding an appearance of an image and aphotographing direction. In addition, the position and attitudecondition may also include another instruction. For example, theposition and attitude condition may include instructions regarding amagnification factor, a distance between the microscope unit and afloor, a distance between the microscope unit and the eye, and/or adirection of the microscope unit 3110 around an optical axis, instead ofor in addition to the above-described appearance of the image and/orphotographing direction.

Furthermore, the instruction included in the position and attitudecondition is registered as a target during control in the abovedescription. In the case of an instruction regarding a photographingdirection, for example, a target of the photographing direction, such asa “vertically downward” direction, is registered. However, the first andsecond embodiments are not limited thereto, and the instruction includedin the position and attitude condition may be registered as arestriction during control. A restriction on a photographing direction,for example, “the optical axis of the microscope unit 3110 should be inthe range of 10 degrees in a vertically downward direction” may beregistered as an instruction regarding a photographing direction.

In addition, the instruction included in the position and attitudecondition may be, for example, an instruction that a specific index bemaximized or minimized (i.e., a condition that prescribes a position andan attitude of the microscope unit 3110 with respect to an operatingsite to obtain a desired captured image at which the specific index hasa maximum or minimum value). For example, the instruction included inthe position and attitude condition may be maximization oftransillumination. In this case, in the drive control systems 1 and 2,luminance of reflected light on a retina may be calculated using acaptured image photographed by the microscope unit 3110, and maximizedtransillumination may be determined to be realized when the luminancehas a maximum value. Thus, the drive control unit 144 illustrated inFIGS. 3 and 13 specifically updates a position and an attitude of themicroscope unit 3110 so that the case of the maximized luminance issearched for while changing a current position and attitude of themicroscope unit 3110 in a narrow range, and thus can realize initialoperation control and trace operation control in which maximizedtransillumination is realized. Note that a degree of transilluminationand a positional relationship between an eye axis and the optical axisof the microscope unit 3110 are considered to have a predeterminedcorrelation. Thus, when the position and attitude of the microscope unit3110 at which the above-described luminance has a maximum value aresearched for, the microscope unit 3110 may be moved within the range ofa predetermined angle (e.g., 15 degrees or narrower) in which an angleformed by the eye axis and the optical axis of the microscope unit 3110falls.

Alternatively, since a degree of transillumination and a positionalrelationship between the eye axis and the optical axis of the microscopeunit 3110 are considered to have a predetermined correlation asdescribed above, the positional relationship between the eye axis andthe optical axis of the microscope unit 3110 in which transilluminationhas a maximum value may be obtained in advance, and the drive controlunit 144 may update the position and attitude of the microscope unit3110 so that the positional relationship between the eye axis and theoptical axis of the microscope unit 3110 in which transillumination hasa maximum value is realized at the time of initial operation control andtrace operation control. Here, it is known that a positionalrelationship between an eye axis and an optical axis of a camera whenthe camera captures the eye can be normally obtained using a positionalrelationship between a corneal ring portion and a position of areflection image of light radiated from a point light source, which isdisposed at a known position of the camera, on a cornea in a capturedimage in which the eye is photographed while the point light sourceradiates light on the eye. Thus, the above-described control can beperformed by obtaining the positional relationship between the eye axisand the optical axis of the microscope unit 3110 from the image by themicroscope unit 3110 using the above-described method.

Note that, in the case in which the instruction included in the positionand attitude condition indicates maximization of transillumination, aninstruction regarding a size of the eye may as well be included in theposition and attitude condition. As the instruction regarding a size ofthe eye is included, the microscope unit 3110 can be moved to makesubstantially no change in the size of the eye in a captured image whenthe position and attitude of the microscope unit 3110 are updated tomaximize transillumination, and thus a further stable image can beprovided to a user, without a significant change in display of thecaptured image.

In addition, at this time, it is desirable to express the instructionregarding the size of the eye with a one-dimensional index such as alength of the eye in the lateral direction or a long axis of a circlecorresponding to the corneal ring portion, rather than a two-dimensionalindex (e.g., a size of a circle corresponding to the corneal ringportion). When the position and attitude of the microscope unit 3110 areupdated to maximize transilluminance, it is assumed to move themicroscope unit 3110 to change the positional relationship between theeye axis and the optical axis of the microscope unit 3110, i.e., tochange the photographing direction; however, if the instructionregarding the size of the eye is expressed with a two-dimensional index,the instruction essentially includes an instruction regarding thephotographing direction as well, and thus there is a possibility of theinstruction regarding the size of the eye limiting movement of themicroscope unit 3110 which is likely to change the photographingdirection. With regard to this matter, if the instruction regarding thesize of the eye is expressed with a one-dimensional index, themicroscope unit 3110 can be moved to change the photographing directionwhile the size of the eye in the captured image is substantiallyuniformly maintained, and therefore the position and attitude of themicroscope unit 3110 at which transillumination has a maximum value canbe more smoothly searched for.

Note that, in a case in which an instruction included in a position andattitude condition is to maximize or minimize a specific index, like theabove-described maximization of transillumination, the trace operationcontrol may be preferably performed. When the trace operation control isperformed and the eye moves, the microscope unit 3110 is automaticallymoved to maximize or minimize the specific index designated by the usertracing the movement of the eye, and thus the user can obtain a desiredcaptured image for which the specific index is maximize or minimized atall times.

(3-3. Other Example of Registration Method of Instruction RegardingAppearance of Image)

The instruction regarding the appearance of the image is registeredusing the GUI illustrated in FIGS. 5 to 7 in the above description.However, the first and second embodiments are not limited thereto, andother GUIs may be used in registration of the instruction regarding theappearance of the image. Note that control over display with respect toeach of the GUIs which will be described below can be executed by thecontrol units 140 and 150 of the drive control systems 1 and 2illustrated in FIGS. 3 and 13. In addition, a display screen on whichthe GUIs are displayed may be the display screen of the display deviceconstituting the pre-operation information acquisition unit 110, or adisplay screen of a separate display device provided in the drivecontrol systems 1 and 2.

FIGS. 16 and 17 are diagrams illustrating an example of another GUI forregistering an instruction regarding an appearance of an image. Asillustrated in FIGS. 16 and 17, a mark 219 indicating the center of acornea and an arrow 209 indicating a vertex direction are displayed on adisplay screen 205 for the GUI. The arrow 209 is a similar one to thearrow 209 of the GUI illustrated in FIGS. 5 to 7. A user can designate acenter position of a corneal ring portion in a captured image and thevertex direction in the captured image by adjusting a position of themark 219 and a direction of the arrow 209. Designation details can beregistered as instructions regarding an appearance of an image. In theillustrated example, the display example of FIG. 16 corresponds to anappearance of an image with respect to upward incision. The displayexample illustrated in FIG. 17 corresponds to an appearance of an imagewith respect to an incision to an ear side. When the user designates thecenter of the cornea and the vertex direction as described above, theinstruction regarding the appearance of the image may be registered.Note that, since a size of the eye image is not stipulated in theinstruction details using the GUI, instruction details in which the sizeof the eye image can be stipulated can be separately registered as aposition and attitude condition by the user. As instructions in whichthe size of the eye image can be stipulated, for example, amagnification factor of a captured image, and a distance between themicroscope unit 3110 and the eye are exemplified.

FIGS. 18 and 19 are diagrams illustrating an example of still anotherGUI for registering the instruction regarding the appearance of theimage. As illustrated in FIGS. 18 and 19, a circle 207 indicating acorneal ring portion is displayed on the display screen 205 for the GUI.The circle 207 is a similar one to the circle 207 of the GUI illustratedin FIGS. 5 to 7. The user can designate a position of the corneal ringportion in a captured image, a size of the corneal ring portion in thecaptured image, and a shape (a photographing direction) of the cornealring portion in the captured image by adjusting a position, a size and aroundness of the circle 207. The designated items can be registered asinstructions regarding appearances of the image. The instructionsregarding the appearances of the image may be registered by designatingthe position, the size, and the shape of the corneal ring portion asdescribed above. Note that, since the vertex direction is not stipulatedin the instruction details of the GUI in this case, instruction detailsin which the vertex direction can be stipulated can be separatelyregistered as a position and attitude condition by the user. As aninstruction in which the vertex direction can be stipulated, forexample, a direction of the microscope unit 3110 around the optical axisis exemplified.

FIG. 20 is a diagram illustrating an example of still another GUI forregistering the instruction regarding the appearance of the image. Asillustrated in FIG. 20, an actual captured image of the eye 213 of thepatient 3405 photographed during a past operation is displayed on thedisplay screen 205 for the GUI. In addition, an arrow 209 indicating avertex direction is displayed being superimposed on the captured image.The arrow 209 is a similar one to the arrow 209 in the GUI illustratedin FIGS. 5 to 7. A size and a position of an image of the eye 213 of thecaptured image can be appropriately modified in accordance with amanipulation of the user, and the user can designate a position of theeye in a captured image that he or she wants to obtain during anupcoming operation, a size of the eye in the captured image by adjustingthe position and the size of the eye 213. In addition, the user candesignate the vertex direction in the captured image that he or shewants to obtain during the upcoming operation by adjusting a directionof the arrow 209. These designated items can be registered asinstructions regarding appearances of the image. The instructionsregarding the appearances of the image may be registered using the GUIfor displaying the actual captured image as described above. Using theactual captured image, the user can designate the appearance of theimage that he or she wants to obtain during the actual operation to meethis or her assumption and register the designated item as a position andattitude condition. Thus, a desired image can be more surely provided tothe user after initial operation control or trace operation control, andthus user convenience can be improved. Note that, an image generatedusing computer graphics that approximates to the actual eye-capturingimage may instead be used, and convenience of this case can be similarlyimproved to the case in which the actual eye-capturing image is used.

(3-4. Other Example of Designation Method for Position and AttitudeCondition)

The position and attitude conditions are designated using the GUIillustrated in FIG. 4 in the above description. The first and secondembodiments, however, are not limited thereto and the position andattitude condition may be designated using another method.

For example, before a operation starts, a user may acquire a capturedimage by controlling a position and an attitude of the microscope unit3110 using his or her hand in a state in which the patient 3405 is lyingdown on his or her left or right side on the patient bed 3403 anddesignate the acquired captured image as an image that the user wants toobtain during the operation. That is, in the present modified example,the pre-operation information acquisition unit 110 illustrated in FIGS.3 and 13 may include the microscope unit 3110 and an appearance (aposition of a corneal ring portion, a size of the corneal ring portion,a vertex direction, and the like) and a photographing direction in acaptured image, and the like may be designated as position and attitudeconditions on the basis of the image captured by the microscope unit3110. According to the present modified example, since the position andattitude conditions are designated for the captured image when anoperator actually moves the microscope unit 3110, position and attitudeconditions can be designated so that a captured image that meets theintension of the user is obtained.

However, in the case in which the position and attitude conditions aredesignated by manually controlling the position and the attitude of themicroscope unit 3110 so that the operator obtains a desired capturedimage as in the present modified example, it is not necessary to performinitial operation control. Thus, a method for designating the positionand attitude conditions according to the present modified example can beappropriately applied when trace operation control is executed.

Note that, in a case in which the method for designating the positionand attitude conditions according to the present modified example isapplied, the designated position and attitude conditions (i.e., positionand attitude conditions based on a captured image acquired through amanipulation of the operator) may be registered in the drive controlsystems 1 and 2 as prescribed position and attitude conditions that canbe used in the future. That is, registration of the position andattitude conditions may be performed using a similar method to themethod for designating the position and attitude conditions according tothe present modified example. In addition, a registered position andattitude condition may be updated with a position and attitude conditionbased on a captured image acquired through a manipulation of theoperator by combining the method for designating the position andattitude conditions according to the present modified example with thelearning process described in (3-1. Updating of position and attitudecondition through learning).

(3-5. Restriction on Movement of Microscope Unit)

Control that the microscope unit 3110 is automatically moved so that theposition and attitude condition is satisfied is performed in the initialoperation control according to the first embodiment and the traceoperation control according to the 20 second embodiment. Various kindsof restrictions may be set on the movement of the microscope unit 3110in the initial operation control and the trace operation control fromthe perspective of safety in the first and second embodiments.

As such restrictions, for example, setting a lower limit value for adistance between the microscope unit 3110 and an eye, setting a limit ona movable range of the arm unit 3120, setting an upper limit value formovement speed of the microscope unit 3110, and/or limiting joint unitsto be driven among the joint units of the arm unit 3120 (e.g., drivingonly several joint units at the leading end side to prevent an attitudeof the arm unit 3120 from significantly changing, or the like), and thelike are considered.

4. Supplement

The preferred embodiment(s) of the present disclosure has/have beendescribed above with reference to the accompanying drawings, whilst thepresent disclosure is not limited to the above examples. A personskilled in the art may find various alterations and modifications withinthe scope of the appended claims, and it should be understood that theywill naturally come under the technical scope of the present disclosure.

For example, the case in which the drive control systems 1 and 2 areapplied to the ophthalmic surgery and the operating site is eye has beendescribed in the first and second embodiments above, the presenttechnology is not limited thereto. For example, the drive controlsystems 1 and 2 may be used in various operations to which themicroscopic operation system 3000 can be applied, such as a laparotomy.In addition, operating sites to be photographed by the microscope unit3110 may be diverse in accordance with operations to which the drivecontrol systems 1 and 2 are applied. In a case in which the drivecontrol systems 1 and 2 are applied to another operation, the term “eye”in the above description of the first and second embodiments may beswitched to biological tissue corresponding to an operating site of theoperation, and processes that constitute the above-described drivecontrol system 1, 2 and performed in the drive control systems 1 and 2may be executed.

Note that, in a case in which the microscopic operation system 3000 isapplied to another operation such as a laparotomy, it is assumed thatthe biological tissue that is an operating site is larger than the eye,and a position of the microscope unit 3110 is somewhat significantlychanged during the operation to facilitate observation of the biologicaltissue in different positions and directions. However, in the initialoperation control and the trace operation control according to theabove-described first and second embodiments, the microscope unit 3110is automatically moved so that the one designated position and attitudecondition is satisfied. Thus, if the initial operation control and thetrace operation control are applied to the other operation, it isnecessary to designate a new position and attitude condition each timethe position of the microscope unit 3110 is changed. Thus, manuallymoving the microscope unit 3110 may sometimes be considered to besimple.

Meanwhile, in a case in which the microscopic operation system 3000 isapplied to an ophthalmic surgery, an eye that is an operating site andthe microscope unit 3110 are considered to have a substantially fixedpositional relationship during the operation. That is, it is assumedthat no major movement would be made once the microscope unit 3110 canbe controlled to have a position and an attitude at which a desiredeye-captured image is obtained. Therefore, the initial operation controland the trace operation control according to the first and secondembodiments are considered to be well compatible with control over aposition and an attitude of the microscope unit 3110 in an ophthalmicsurgery in that the microscope unit 3110 is automatically moved so thatthe one designated position and attitude condition is satisfied.

In addition, it is assumed in the ophthalmic surgery that an image isphotographed at a fairly high magnification factor to observe the eyethat is a fairly small operating site. In the case in whichphotographing is performed with a high magnification factor, a capturedimage is significantly changed only by slightly changing a position andan attitude of the microscope unit 3110, and thus it is necessary tocontrol the position and the attitude of the microscope unit 3110 withhigh precision, and manual manipulations are difficult in most cases. Onthe other hand, since a position and an attitude of the microscope unit3110 can be automatically controlled in the initial operation controland the trace operation control according to the first and secondembodiments, this system is particularly suitable for such a case inwhich photographing with a high magnification factor is required.

Furthermore, in the trace operation control according to the secondembodiment, a position and an attitude of the microscope unit 3110 areautomatically controlled so that the designated position and attitudecondition is satisfied while movement of the operating site is traced.Meanwhile, in a operation such as a laparotomy other than ophthalmicsurgeries, it is considered that a biological tissue that is anoperating site is fairly large as described above and movement thereofduring the operation is greater than in the ophthalmic surgeries, andthus if the microscope unit 3110 is moved tracing the movement of thebiological tissue, the amount of movement of the microscope unit 3110increases accordingly. On the other hand, since the eye is fairly smallas an operating site and movement thereof during the operation is alsosmall, even if the microscope unit 3110 is moved tracing the movement ofthe biological tissue, the amount of movement of the microscope unit3110 is suppressed in a small range. Thus, safety can be secured even ifthe microscope unit 3110 is automatically moved.

Taking account of the above-described circumstances, the drive controlsystems 1 and 2 according to the first and second embodiments areconsidered to exhibit favorable effect when they are applied toophthalmic surgeries.

Note that the term “corneal ring portion” used in the above descriptionhas substantially the same definition as terms “corneal boundary,”“corneal range,” and the like. In addition, for a term indicating avertex direction, various terms that are generally used, such as“upward,” “downward,” “nose side,” “ear side,” and the like can be used.The various terms used in the above description can appropriately readas other terms that can be easily understood by a person skilled in theart.

Further, the effects described in this specification are merelyillustrative or exemplified effects, and are not limitative. That is,with or in the place of the above effects, the technology according tothe present disclosure may achieve other effects that are clear to thoseskilled in the art from the description of this specification.

Additionally, the present technology may also be configured as below.

(1)

A control device including:

a control unit configured to control a position and an attitude of amicroscope unit by driving an arm unit that supports the microscope uniton the basis of a captured image of an operating site photographed bythe microscope unit during an operation so that a position and attitudecondition set before the operation is satisfied,

in which the position and attitude condition is a condition thatprescribes a position and an attitude of the microscope unit withrespect to the operating site to obtain a desired captured imagecorresponding to the position and attitude condition.

(2)

The control device according to (1), in which the position and attitudecondition at least includes an instruction regarding an appearance ofthe image of the operating site.

(3)

The control device according to (2), in which the instruction regardingthe appearance of the image of the operating site includes at least oneof details of a position of the operating site in the captured image,details of a size of the operating site in the captured image, detailsof a shape of the operating site in the captured image, and details of avertex direction in the captured image.

(4)

The control device according to any one of (1) to (3), in which theposition and attitude condition includes at least one of an instructionregarding a photographing direction, an instruction regarding amagnification of the captured image, an instruction regarding a distancebetween the microscope unit and a floor, and an instruction regarding adistance between the microscope unit and the operating site.

(5)

The control device according to any one of (1) to (4), in which theoperating site is an eye.

(6)

The control device according to (5),

in which the position and attitude condition at least includes aninstruction regarding an appearance of an image of the eye, and

the instruction regarding the appearance of the image of the eyeincludes at least one of details of a position of a corneal ring part ofthe eye in the captured image, details of a size of the corneal ringpart of the eye in the captured image, details of a shape of the cornealring part of the eye in the captured image, and details of a vertexdirection in the captured image.

(7)

The control device according to any one of (1) to (6), in which theposition and attitude condition includes an instruction to maximize orminimize a specific index.

(8)

The control device according to (7),

in which the operating site is an eye,

the specific index is transillumination, and

an instruction to maximize the transillumination includes thatbrightness of reflection light from a retina be at a maximum or that aneye axis of the eye and an optical axis of the microscope unit have apredetermined positional relationship.

(9)

The control device according to any one of (1) to (8), in which theposition and the attitude of the microscope unit are controlled untilthe captured image photographed during the operation approximates thedesired captured image corresponding to the position and attitudecondition set before the operation.

(10)

The control device according to (9), in which whether the captured imagephotographed during the operation approximates the desired capturedimage corresponding to the position and attitude condition set beforethe operation is determined by comparing feature amounts extracted fromthe images.

(11)

The control device according to any one of (1) to (8), in which theposition and the attitude of the microscope unit are controlled so thatthe position and attitude condition set before the operation issatisfied until an end instruction is input.

(12)

The control device according to any one of (1) to (11), in which the armunit which supports the microscope unit is driven and the position andthe attitude of the microscope unit are controlled so that the positionand attitude condition is satisfied on the basis of a captured imageoverlooking a vicinity of the operating site including the operatingsite.

(13)

The control device according to any one of (1) to (12),

in which one position and attitude condition among position and attitudeconditions that are registered in advance is set, and

the position and the attitude of the microscope unit are controlled sothat the set position and attitude condition is satisfied.

(14)

The control device according to (13),

in which, in a case in which the position and the attitude of themicroscope unit are modified after the position and the attitude of themicroscope unit are controlled so that the set position and attitudecondition is satisfied, registration details of the set position andattitude condition are updated on the basis of the modified position andattitude of the microscope unit.

(15)

The control device according to (13) or (14), in which the control unitcauses an icon indicating the position and attitude condition that isregistered in advance to be displayed on a display screen.

(16)

A control method including:

controlling, by a processor, a position and an attitude of a microscopeunit by driving an arm unit that supports the microscope unit on thebasis of a captured image of an operating site photographed by themicroscope unit during an operation so that a position and attitudecondition set before the operation is satisfied,

in which the position and attitude condition is a condition thatprescribes a position and an attitude of the microscope unit withrespect to the operating site to obtain a desired captured imagecorresponding to the position and attitude condition.

(17)

A microscope device for operation including:

a microscope unit configured to photograph a captured image of anoperating site;

an arm unit configured to support the microscope unit; and

a control device configured to control a position and an attitude of themicroscope unit by driving the arm unit on the basis of a captured imageof the operating site photographed by the microscope unit during anoperation so that a position and attitude condition set before theoperation is satisfied,

in which the position and attitude condition is a condition thatprescribes a position and an attitude of the microscope unit withrespect to the operating site to obtain a desired captured imagecorresponding to the position and attitude condition.

(16)

A control method including controlling, by a processor, a position andan attitude of a microscope unit by driving an arm unit that supportsthe microscope unit on the basis of a captured image of an operatingsite photographed by the microscope unit during an operation so that aposition and attitude condition set before the operation that is acondition that prescribes a position and an attitude of the microscopeunit with respect to the operating site to obtain a desired capturedimage is satisfied.

(17)

A microscope device for operation including:

a microscope unit configured to photograph a captured image of anoperating site;

an arm unit configured to support the microscope unit; and

a control device configured to control a position and an attitude of themicroscope unit by driving the arm unit on the basis of a captured imageof the operating site photographed by the microscope unit during anoperation so that a position and attitude condition set before theoperation that is a condition that prescribes a position and an attitudeof the microscope unit with respect to the operating site to obtain adesired captured image is satisfied.

REFERENCE SIGNS LIST

-   1, 2 drive control system-   110 pre-operation information acquisition unit-   120 in-operation information acquisition unit-   130 driving unit-   140, 150 control unit-   141 position and attitude condition setting unit-   142 state recognition unit-   143 state comparison unit-   144 drive control unit-   153 end determination unit-   3000 microscopic operation system-   3100 microscope device-   3110 microscope unit-   3120 arm unit-   3121 a first joint unit-   3121 b second joint unit-   3121 c third joint unit-   3121 d fourth joint unit-   3121 e fifth joint unit-   3121 f sixth joint unit-   3123 a first ring-   3123 b second ring-   3123 d third ring-   3123 d fourth ring-   3123 e fifth ring-   3123 f sixth ring-   3130 base unit-   3200 control device-   3300 display device

The invention claimed is:
 1. A surgical imaging system comprising: amemory configured to store a plurality of position and attitudeconditions each corresponding to a fixed position and attitude of amicroscope; display circuitry configured to cause a plurality of imagesto be displayed simultaneously, each of the plurality of imagescorresponding to a different position and attitude condition in theplurality of position and attitude conditions; input circuitryconfigured to receive a user input operation to select, from thedisplayed plurality of images, a first displayed image corresponding toa first position and attitude condition in the plurality of storedposition and attitude conditions and corresponding to a first fixedposition and attitude of the microscope to capture a first image of asurgical site by the microscope; an arm configured to support themicroscope and adjust a position and attitude of the microscope to be asecond fixed position and attitude that is different than the firstfixed position and attitude; the microscope configured to capture asecond image of the surgical site from the second fixed position andattitude; and processing circuitry configured to control, in response tothe user input operation, a position and an attitude of the microscopeto be changed from the second fixed position and attitude to the firstfixed position and attitude by driving the arm based on the stored firposition and attitude condition so that a third image of the surgicalsite, captured by the microscope, corresponds to the first image.
 2. Thesurgical imaging system according to claim 1, wherein the first positionand attitude condition is stored in association with a user of themicroscope.
 3. The surgical imaging system according to claim 1, whereinthe first position and attitude condition is stored in association witha kind of surgical procedure.
 4. The surgical imaging system accordingto claim 1, wherein the processing circuitry is configured to drive thearm so that an optical axis of the microscope stays pointed at a certainpoint in space.
 5. The surgical imaging system according to claim 1,wherein the microscope includes an imaging device configured to generatean image signal corresponding to stereoscopic vision.
 6. The surgicalimaging system according to claim 1, wherein the microscope includes animaging device having an auto-focus (AF) function.
 7. The surgicalimaging system according to claim 1, further comprising a display deviceconfigured to display the third image of the surgical site captured bythe microscope.
 8. The surgical imaging system according to claim 1,wherein the first position and attitude condition includes aninstruction regarding an appearance of the first image of the surgicalsite.
 9. The surgical imaging system according to claim 8, wherein theinstruction regarding the appearance of the first image of the surgicalsite includes at least one of: details of a position of the surgicalsite in the first image, details of a size of the surgical site in thefirst image, details of a shape of the surgical site in the first image,and details of a vertex direction in the first image.
 10. The surgicalimaging system according to claim 1, wherein the first position andattitude condition includes at least one of: an instruction regarding animage capturing direction, an instruction regarding a magnification ofthe first image, an instruction regarding a distance between themicroscope and a floor, and an instruction regarding a distance betweenthe microscope and the surgical site.
 11. The surgical imaging systemaccording to claim 1, wherein the surgical site is an eye.
 12. Thesurgical imaging system according to claim 11, wherein the firstposition and attitude condition includes an instruction regarding anappearance of the first image of the eye, and the instruction regardingthe appearance of the first image of the eye includes at least one of:details of a position of a corneal ring part of the eye in the firstimage, details of a size of the corneal ring part of the eye in thefirst image, details of a shape of the corneal ring part of the eye inthe first image, and details of a vertex direction in the first image.13. The surgical imaging system according to claim 1, wherein the firstposition and attitude condition includes an instruction to maximize orminimize an index.
 14. The surgical imaging system according to claim13, wherein the surgical site is an eye, the index is transillumination,and an instruction to maximize the transillumination includes thatbrightness of reflection light from a retina be at a maximum or that aneye axis of the eye and an optical axis of the microscope have apredetermined positional relationship.
 15. The surgical imaging systemaccording to claim 1, wherein the position and the attitude of themicroscope are controlled until the third image captured during asurgical operation sufficiently corresponds to the first imagecorresponding to the first position and attitude condition set beforethe surgical operation.
 16. The surgical imaging system according toclaim 15, wherein whether the third image captured during the surgicaloperation sufficiently corresponds to the first image, corresponding tothe position and attitude condition set before the surgical operation,is determined by comparing feature amounts extracted from the first andthird images.
 17. The surgical imaging system according to claim 1,wherein the position and the attitude of the microscope are controlledso that the first position and attitude condition set before a surgicaloperation is satisfied until an end instruction is input.
 18. Thesurgical imaging system according to claim 1, wherein the arm whichsupports the microscope is driven and the position and the attitude ofthe microscope are controlled so that the first position and attitudecondition is satisfied on the basis of a fourth image overlooking avicinity of the surgical site and includes the surgical site.
 19. Thesurgical imaging system according to claim 1, wherein one position andattitude condition, among the plurality of position and attitudeconditions that are registered in advance, is pre-set, and theprocessing circuitry is further configured to control the position andthe attitude of the microscope so that the one position and attitudecondition is satisfied.
 20. The surgical imaging system according toclaim 19, wherein, in a case that the position and the attitude of themicroscope are modified after the position and the attitude of themicroscope were controlled by the processing circuitry so that the firstposition and attitude condition was satisfied, registration details ofthe one position and attitude condition are updated on the basis of themodified position and attitude of the microscope.
 21. The surgicalimaging system according to claim 19, wherein the plurality of imagesare displayed on a display screen.
 22. A method comprising: storing aplurality of position and attitude conditions, each corresponding to afixed position and attitude of a microscope; displaying simultaneously aplurality of images, each corresponding to a different position andattitude condition in the plurality of position and attitude conditions;receiving a user input operation to select, from the displayed pluralityof images, a first displayed image corresponding to a first position andattitude condition in the plurality of stored position and attitudeconditions and corresponding to a first fixed position and attitude ofthe microscope to capture a first image of a surgical site by themicroscope, in a memory; adjusting a position and attitude of themicroscope to be a second fixed position and attitude that is differentthan the first fixed position and attitude; capturing a second image ofthe surgical site with the microscope supported by an arm from thesecond fixed position and attitude; and controlling, using processingcircuitry and in response to the user input operation, a position and anattitude of the microscope to be changed from the second fixed positionand attitude to the first fixed position and attitude by driving the armbased on the stored first position and attitude condition so that athird image of the surgical site, captured by the microscope,corresponds to the first image.
 23. An apparatus comprising: processingcircuitry configured to control, in response to a user input operationreceived by input circuitry, a position and an attitude of a microscopeconfigured to capture a third image of a surgical site by driving an armconfigured to support the microscope from a second fixed position andattitude of the microscope where a second image is captured, based on afirst position and attitude condition stored in a memory andcorresponding to a first fixed position and attitude of the microscopeused to capture a first image of the surgical site by the microscope,the first position and attitude condition being selected from aplurality of position and attitude conditions stored in the memory byinput circuitry that selects a first displayed image corresponding tothe first position and attitude condition from a plurality ofsimultaneously displayed images, so that the third image of the surgicalsite, captured by the microscope, corresponds to the first image, andwherein the second fixed position and attitude of the microscope isdifferent from the first fixed position and attitude of the microscope.24. The surgical imaging system according to claim 1, wherein each ofthe stored plurality of position and attitude conditions furtherincludes focus and magnification information corresponding to the firstfixed position and attitude of the microscope to capture the first imageof the surgical site by the microscope, and the processing circuitry isfurther configured to control, in response to the user input operation,a focus and magnification of the microscope to capture the third image,based on the stored first condition information.
 25. The surgicalimaging system according to claim 1, wherein each of the storedplurality of position and attitude conditions further includes distanceinformation between the microscope and the surgical site correspondingto the first fixed position and attitude of the microscope to capturethe first image of the surgical site by the microscope, and theprocessing circuitry is further configured to control, in response tothe user input operation, the focus and magnification of the microscopeto capture the third image, based on the stored first conditioninformation.
 26. The surgical imaging system according to claim 1,wherein each of the stored plurality of position and attitude conditionsfurther includes instruction information corresponding to the firstfixed position and attitude of the microscope to capture the first imageof the surgical site by the microscope, the instruction informationindicating whether to maximize a brightness of reflection light from aportion of the surgical site, and the processing circuitry is furtherconfigured to control, in response to the user input operation,execution of the instruction information in the stored first conditioninformation to capture the third image.
 27. The surgical imaging systemaccording to claim 1, wherein each of the plurality of images to bedisplayed includes an icon.